Evaluating Neural Machine Translation in English-Japanese Task

In this paper, we evaluate Neural Machine Translation (NMT) models in English-Japanese translation task. Various network architectures with different recurrent units are tested. Additionally, we examine the effect of using pre-reordered data for the training. Our experiments show that even simple NMT models can produce better translations compared with all SMT baselines. For NMT models, recovering unknown words is another key to obtaining good translations. We describe a simple workaround to find missing translations with a back-off system. To our surprise, performing prereordering on the training data hurts the model performance. Finally, we provide a qualitative analysis demonstrates a specific error pattern in NMT translations which omits some information and thus fail to preserve the complete meaning.